{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "23af4ff9-2300-458b-9486-9278cb2abea5",
   "metadata": {
    "tags": []
   },
   "source": [
    "# EasyCV多目标跟踪\n",
    "\n",
    "本文将介绍如何使用EasyCV快速使用FCOS检测器和ReID进行多目标跟踪的训练、推理"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c93bdca-6cb5-451e-92fd-653e33d2aa83",
   "metadata": {
    "tags": []
   },
   "source": [
    "## 安装依赖包\n",
    "\n",
    "注: 在PAI-DSW docker中无需安装相关依赖，可跳过此部分 在本地notebook环境中执行\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "846076f6-ac34-49c9-aaff-e82aecd6336d",
   "metadata": {},
   "source": [
    "1、 首先，安装pytorch和对应版本的torchvision，支持Pytorch1.5.1以上版本"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "259487df-b711-4457-ba0a-b32caf34079a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# install pytorch and torch vision\n",
    "! conda install --yes pytorch==1.10.0 torchvision==0.11.0 -c pytorch"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "323f9718-927e-4377-aa80-cf5c398d54c5",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true,
    "tags": []
   },
   "source": [
    "2、 获取torch和cuda版本，安装对应版本的mmcv和nvidia-dali"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "57f87096-7801-4254-9dfd-cbdb4ab54cfc",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "import os\n",
    "os.environ['CUDA']='cu' + torch.version.cuda.replace('.', '')\n",
    "os.environ['Torch']='torch'+torch.version.__version__.replace('+PAI', '')\n",
    "!echo $CUDA\n",
    "! echo $Torch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "99e24dec-b747-4a23-a981-9aae14e433cf",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# install some python deps\n",
    "! pip install mmcv-full==1.4.4 -f https://download.openmmlab.com/mmcv/dist/${CUDA}/${Torch}/index.html\n",
    "! pip install http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/third_party/nvidia_dali_cuda100-0.25.0-1535750-py3-none-manylinux2014_x86_64.whl"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6166dd4e-0e31-4061-9229-5f79968732e0",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true,
    "tags": []
   },
   "source": [
    "3、  安装EasyCV算法包"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d55d2b68-b9d3-484a-abf6-561187153c54",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "pip install pai-easycv"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5e4d314-f59f-464c-950d-12c94b11020c",
   "metadata": {
    "tags": []
   },
   "source": [
    "## 简单验证"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "80c51674-b1cc-4acf-a8e3-ed20306afebf",
   "metadata": {},
   "outputs": [],
   "source": [
    "from easycv.apis import *"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0512498b-a900-4969-9563-67029a1009e2",
   "metadata": {
    "tags": []
   },
   "source": [
    "## 运行环境要求\n",
    "\n",
    "PAI-Pytorch 1.7/1.8镜像， GPU机型 P100 or V100， 内存 32G"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e769ad6-3ae0-4d5b-bf23-55673a239aa8",
   "metadata": {},
   "source": [
    "### 数据准备\n",
    "先通过docs/source/data_hub.md下载crowdhuman、mot20和Market1501三个数据集，然后使用tools/prepare_data/目录下的crowdhuman2coco.py、mot2coco.py和prepare_market1501.py文件将三个数据集转化成coco格式\n",
    "\n",
    "data/coco格式如下\n",
    "\n",
    "```shell\n",
    "data/coco/\n",
    "├── annotations\n",
    "│   ├── train.json\n",
    "│   └── val.json\n",
    "├── train\n",
    "│   ├── 000000005802.jpg\n",
    "│   ├── 000000060623.jpg\n",
    "│   ├── 000000086408.jpg\n",
    "│   ├── 000000118113.jpg\n",
    "│   ├── 000000184613.jpg\n",
    "│   ├── 000000193271.jpg\n",
    "│   ├── 000000222564.jpg\n",
    "│       ...\n",
    "│   └── 000000574769.jpg\n",
    "└── val\n",
    "    ├── 000000006818.jpg\n",
    "    ├── 000000017627.jpg\n",
    "    ├── 000000037777.jpg\n",
    "    ├── 000000087038.jpg\n",
    "    ├── 000000174482.jpg\n",
    "    ├── 000000181666.jpg\n",
    "    ├── 000000184791.jpg\n",
    "    ├── 000000252219.jpg\n",
    "         ...\n",
    "    └── 000000522713.jpg\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed28f2be-1519-4d81-9d03-42891d1e9bc7",
   "metadata": {},
   "source": [
    "### 训练FCOS检测模型\n",
    "下载训练配置文件"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bd8b6d19-652c-4a58-89d3-38394dbb9a20",
   "metadata": {},
   "outputs": [],
   "source": [
    "! rm -rf fcos_r50_torch_1x_mot20_crowdhuman.py\n",
    "!wget https://raw.githubusercontent.com/alibaba/EasyCV/master/configs/detection/fcos/fcos_r50_torch_1x_mot20_crowdhuman.py"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d2ed375-f5ef-475e-9dc8-3386d328b8b0",
   "metadata": {},
   "source": [
    "使用单卡gpu进行训练和验证集评估"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9b6ec6da-d125-4ebe-b301-0d8cc8881512",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "! python -m easycv.tools.train  fcos_r50_torch_1x_mot20_crowdhuman.py --work_dir work_dirs/mot/fcos/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82b04e66-ef62-4d2e-963f-34015a588786",
   "metadata": {},
   "source": [
    "### 训练ReID模型\n",
    "下载训练配置文件"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1d4f7868-88aa-428c-9852-22e3eca45a07",
   "metadata": {},
   "outputs": [],
   "source": [
    "! rm -rf market1501_resnet50_jpg.py\n",
    "!wget https://raw.githubusercontent.com/alibaba/EasyCV/master/configs/classification/imagenet/resnet/market1501_resnet50_jpg.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "67aa8262-834a-4afc-a6cd-fd882a181371",
   "metadata": {},
   "outputs": [],
   "source": [
    "! python -m easycv.tools.train  market1501_resnet50_jpg.py --work_dir work_dirs/mot/reid/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dc8be915",
   "metadata": {},
   "outputs": [],
   "source": [
    "评估ReID模型(checkpoint路径替换成work_dirs/mot/reid/epoch_60_export.pt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8d06f30c",
   "metadata": {},
   "outputs": [],
   "source": [
    "! python -m unittest tests/predictors/test_reid_predictor.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f9d31ac1",
   "metadata": {},
   "outputs": [],
   "source": [
    "使用预先训练好的reid模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c08d7e7",
   "metadata": {},
   "outputs": [],
   "source": [
    "! python easycv/thirdparty/reid/inference.py --checkpoint https://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/data/tracking/reid_r50_epoch_60_export.pt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72d78ef4-0d90-4f3a-a1d9-c532e17d065b",
   "metadata": {},
   "source": [
    "### 预测\n",
    "下载测试视频"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "732fa5b4-fe0f-4a9b-ba19-d73d3ebd1736",
   "metadata": {},
   "outputs": [],
   "source": [
    "! wget https://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/data/tracking/MOT20-01.zip && unzip MOT20-01.zip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cb3b24c2-7a19-4618-8bdf-42ebe1cdfeb3",
   "metadata": {},
   "outputs": [],
   "source": [
    "单镜头MOT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5e617239-b034-4760-8c55-326cf3a6e475",
   "metadata": {},
   "outputs": [],
   "source": [
    "! python easycv/thirdparty/mot/demo_mot.py --input MOT20-01/img1 --output work_dirs/mot/result.mp4 --checkpoint work_dirs/mot/fcos/epoch_12_export.pt --fps 24"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3dfc44c7",
   "metadata": {},
   "outputs": [],
   "source": [
    "使用预先训练好的fcos模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "45b7f4d3",
   "metadata": {},
   "outputs": [],
   "source": [
    "! python easycv/thirdparty/mot/demo_mot.py --input MOT20-01/img1 --output work_dirs/mot/result.mp4 --checkpoint https://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/data/tracking/fcos_r50_epoch_12_export.pt --fps 24"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "63565edf",
   "metadata": {},
   "outputs": [],
   "source": [
    "跨镜头MOT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5ae273b9",
   "metadata": {},
   "outputs": [],
   "source": [
    "! python easycv/thirdparty/mot/demo_mtmct.py --det_checkpoint work_dirs/mot/fcos/epoch_12_export.pt --reid_checkpoint work_dirs/mot/reid/epoch_60_export.pt --input mtmct-demo/ --output work_dirs/mot/ --save_images --use_folder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2f18e4cd",
   "metadata": {},
   "outputs": [],
   "source": [
    "使用预先训练好的fcos和reid模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e5b445cb",
   "metadata": {},
   "outputs": [],
   "source": [
    "! python easycv/thirdparty/mot/demo_mtmct.py --det_checkpoint https://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/data/tracking/fcos_r50_epoch_12_export.pt --reid_checkpoint https://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/data/tracking/reid_r50_epoch_60_export.pt --input mtmct-demo/ --output work_dirs/mot/ --save_images --use_folder"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
